Science fiction, in its most perfect form, operates like a Möbius strip. Then, years later, early adherents look back and analyze its predictions, knowing full well that sci-fi set the blueprint for the world they're living in. Rarely, though, do the creators of sci-fi get to revisit the worlds they built after the events they anticipated are set in motion. In this, Lana and Lilly Wachowski are all but singular. When The Matrix came out in 1999, it was a beautifully realized cyberpunk fable.

This paper tackles the problem of learning a questioner in the goal-oriented visual dialog task. Several previous works adopt model-free reinforcement learning. Most pretrain the model from a finite set of human-generated data. We argue that using limited demonstrations to kick-start the questioner is insufficient due to the large policy search space. Inspired by a recently proposed information theoretic approach, we develop two analytic experts to serve as a source of high-quality demonstrations for imitation learning. We then take advantage of reinforcement learning to refine the model towards the goal-oriented objective. Experimental results on the GuessWhat?! dataset show that our method has the combined merits of imitation and reinforcement learning, achieving the state-of-the-art performance.